Learning Preferences and User Engagement Using Choice and Time Data

Choice decisions made by users of online applications can suffer from biases due to the users’ level of engagement. For instance, low engagement users may make random choices with no concern for the quality of items offered. This biased choice data can corrupt estimates of user preferences for items. However, one can correct for these biases if additional behavioral data is utilized. To do this we construct a new choice engagement time model which captures the impact of user engagement on choice decisions and response times associated with these choice decisions. Response times are the behavioral data we choose because they are easily measured by online applications and reveal information about user engagement. To test our model we conduct online polls with subject populations that have different levels of engagement and measure their choice decisions and response times. We have two main empirical findings. First, choice decisions and response times are correlated, with strong preferences having faster response times than weak preferences. Second, low user engagement is manifested through more random choice data and faster response times. Both of these phenomena are captured by our choice engagement time model and we find that this model fits the data better than traditional choice models. Our work has direct implications for online applications. It lets these applications remove the bias of low engagement users when estimating preferences for items. It also allows for the segmentation of users according to their level of engagement, which can be useful for targeted advertising or marketing campaigns.